The present invention relates to methods of encoding an auxiliary signal within a video signal, and more particularly to methods for subliminally modulating a video signal with an auxiliary signal by use of an encoding improvement in a manner that is not only less perceptible or imperceptible to viewers of the video signal but also easier to detect by a detector.
A video signal is modulated to contain an auxiliary signal at an encoder thereby creating a modulated video signal. The modulated video signal is then transmitted from a signal source to either a display device where the auxiliary signal may be optically detected by use of a hand-held optical detector or electrically detected by an inline detector.
Users of these detectors selectively receive auxiliary signals for purposes including enjoyment, promotion, transfer of information, data collection, commercial verification, security, education, and transactions or verifications at points of sale, as well as other commercial, personal, entertainment, or amusement purposes collectively referred to herein as “promotional opportunities”.
U.S. Pat. No. 4,807,031 to Broughton et al. (“Broughton”) entitled “Interactive Video Method and Apparatus” relates generally to in-band video broadcasting of commands and other encoded information to interactive devices. The invention described therein relates generally to interactive educational and entertainment systems, and is described in one embodiment in the context of television program control of toys located where there is a television receiver, as within a residence.
To encode control data capable of providing a benefit to a user, Broughton discloses a novel method of luminance or chrominance modulation of a video signal that creates a modulated video signal, wherein the video signal is modulated with control data. The novel modulation method alternately raises and lowers the luminance/chrominance of adjacent horizontal scan lines to create a video subcarrier that contains the control data.
In Broughton, the video signal is not being replaced with other data, nor is the data being added as a separate signal along with the video signal. Rather, the video signal itself is modulated to carry the control data. Therefore, the control data is a part of, or contained within, the video signal and yet is imperceptible to the human eye and relatively invisible. The encoding method also includes preview and remove circuitry to ensure suitability or the presence of data encoding and removal of data encoding, respectively.
The control data is transmitted either by television broadcast means, or by pre-recorded video players that are connected to a video display. The control data is then received by the video display where at least one video field of the video display is modulated by control data. The control data is then detected with either opto-electronic or radio frequency (RF) detection means that discriminate between the program material and the control data to detect the control data. The detected control data is further reproduced so that the control data can be used with an interactive device.
Improvements on the method of modulation described in Broughton are described in U.S. Pat. No. 6,094,228 to Ciardullo et al. and U.S. Pat. No. 6,229,572 to Ciardullo et al. (referred to collectively herein as “Ciardullo”). Both Ciardullo patents describe improved methods of modulation wherein the auxiliary data is inserted on the visual portion of a video signal by changing the luminance of paired scan lines in opposite directions. Instead of raising and lowering the intensity on the whole line as in Broughton, Ciardullo uses pseudo noise sequences to raise and lower the intensity on portions of a first line, where the line paired to the first line is modulated with the inverse pseudo noise sequences. Ciardullo thereby allows larger amounts of auxiliary data to be modulated in the video signal by use of the pseudo noise sequences. Broughton and Ciardullo, which are owned by the assignee of the present invention, are incorporated by reference herein.
Yet another patent application by the assignees of the present invention is U.S. Patent Application entitled “RBDS Method and Device for Processing Promotional Opportunities”, Ser. No. 10/126,770, filed on Apr. 19, 2002, of James G. Withers and Alan G. Maltagliati (referred to hereinafter as “Withers I”), which is incorporated by reference herein. Withers I describes further improvements to Koplar I and Koplar II including the transmission of auxiliary data to a hand-held device by use of the RBDS system.
Still another patent application by the assignees of the present invention is U.S. Patent Application entitled “Method and Apparatus for Modulating a Video Signal With Data”, Ser. No. 10/676,940, filed on Oct. 1, 2003, of Yousri H. Barsoum, Alan G. Maltagliati, Daniel A. Ciardullo and Christopher E. Chupp (the application being herein referred to as “Barsoum”), which is incorporated by reference herein. Barsoum describes the use of a memory card coupled to a slotted hand-held device for receiving video signals modulated with carrier signals by use of the memory card, transmitting the signals from the card to the slotted hand-held device, detecting the carrier signals or decoding auxiliary data from the video signals on the slotted hand-held device and providing the user of the slotted hand-held device with a benefit from the detection of the carrier signals and/or the reception of the auxiliary data. Barsoum also describes a method of improving the reliability and speed of the transmission and reception of auxiliary data by storing data in video frames split into multiple regions and encoding complementary data bits in each region.
Still another patent application by the assignees of the present invention is U.S. Patent Application entitled “Method And System of Detecting Signal Presence from a Video Signal Presented on a Digital Display Device”, Ser. No. 10/817,109, filed on Apr. 2, 2004, of James G. Withers, Yousri H. Barsoum, Edward J. Koplar and Michael C. Reynolds (the application being herein termed “Withers II”), which is incorporated by reference herein. Withers II describes several methods and apparatus for modulating video signals with signal presence and signal absence for use on digital display devices.
Yet another patent application by the assignees of the present invention is U.S. Patent Application entitled “Method and System for Enhanced Modulation of Video Signals”, Ser. No. 10/888,919, filed on Jul. 9, 2004, of Christopher E. Chupp, Michael S. Gramelspacher, Jesse J. Chounard II, James G. Withers, Yousri H. Barsoum and Michael C. Reynolds (the application being herein termed “Chupp”), which is incorporated by reference herein. Chupp describes a method and apparatus for determining an optimum level and placement of a carrier signal (i.e., an auxiliary signal) to be modulated into an active portion of a video signal so as to deter nefarious third parties from stripping the carrier signal out of the video signal and increase the detectability of the carrier signal within the video signal without noticeably decreasing the clarity of a picture represented by video signal to a viewer.
At the time of the present invention, analog display devices (e.g., NTSC televisions) operate by use of a fine pitch electron beam that strikes phosphors coating on an internal face of the cathode ray tube (CRT). The phosphors emit light of an intensity which is a function of the intensity of the beam striking it. A period of 1/60 second is required for the electron beam to completely scan down the CRT face to display a field of the image. During the following 1/60 second, an interlaced field is scanned, and a complete frame of video is then visible on the analog display device. The phosphors coating on the face of the tube is chemically treated to retain its light emitting properties for a short duration. Thus, the first area of the scanned picture begins to fade just as the electron beam retraces (i.e., during the vertical retrace) to the top of the screen to refresh it. Since the electron beam covers 525 lines 30 times per second, a total of 15,750 lines per second is viewed each second.
Broughton's method of encoding a carrier signal in a video signal and its improvements were generally intended for use with an analog display device. Upon receiving the video signal from the signal source, such a display device splits the video signal into sequentially transmitted images referred to as frames, whereby each frame of an NTSC television image has 525 horizontal scan lines. The display device scans 262.5 of the horizontal lines left to right and top to bottom by skipping every other line, thus completing the scan of a first field, and then retracing to the top of the image and scanning the remaining 262.5 lines, for a second field. The fields are interlaced at the display device and construct one complete frame. When the video signal is broadcast at 525 lines per frame and 30 frames a second there are 60 fields per second and a line frequency rate (i.e., the speed at which lines are refreshed) of 15,750 Hz (i.e., approximately 16 kHz). Despite the success of the technology of Broughton and its improvements, wherein the modulation of video with carrier signals results in at no worse than subliminal visual changes which are substantially invisible, there is a need in the art for a new apparatus and method for modulating a video signal with an auxiliary signal wherein the signal is even more completely invisible and yet more reliably detected.
Although Broughton has been frequently used and well received since its inception, the relative invisibility of the carrier signal in the picture of the display device and the ease of detecting the carrier signal from the display device by a hand-held device or other detector can be yet improved. A slight tendency to visibility may occur when the voltage of the carrier signal is increased for the purpose of increasing the carrier signal's detectability, as television viewers might then slightly perceive the effects of the carrier signal on the visible picture, such as a slight tendency of visible lines or a slight deterioration in the picture quality. Since one of the advantages of using Broughton is its relative invisibility of auxiliary data to the human eyesight, any tendency of viewing the effects of the carrier signal is undesirable.
The invisibility challenge is typically resolved by reducing the voltage (i.e., as resultant luminosity) added to or removed from the selected video scan lines. However, lowering the overall signal intensity decreases the reliability of detecting the carrier signal and creates difficulty in detecting the carrier signal on a hand-held device over a distance from a display device, so that users may not receive the carrier signal, auxiliary data and/or promotional opportunity they expected to receive.
In making use of the present invention, signals are received, detected, and reproduced by the hand-held devices and other receivers for various promotional opportunities including: enjoyment; promotion; coupon or prize validation; advertising by sponsors; advertising verification and polling; transfer of information; data collection; commercial verification; security and access; education; game playing; transactions; verifications; or redemption by sponsoring entities or related commercial locations at points of sale including the Internet; or for other commercial and non-commercial purposes.
For purposes of the present invention, the term “hand-held device” means an interactive device of portable character, preferably of hand-held type that may be carried in the palm by a user, or between fingers of the user, or is otherwise intended to be easily grasped and handled manually by the user. Smart cards, mobile phones, personal digital assistants (PDAs), gaming devices and similar hand-held devices with or without capability for memory cards that are capable of participating with the promotional opportunities described in the present invention and are collectively referred to herein as “hand-held devices” of the present invention.
The term “computer” is also used herein in its broadest possible sense, and may include without limitation a laptop, compact or personal computer, mobile phone, gaming device, personal digital assistant (PDA), or other computer-like device, or other devices using one or more microprocessors or digital processors to achieve a computing or data processing or data manipulative process or comparable or similar functions.